Big Data is basically vast amount of data which cannot be effectively processed, captured, and analyzed by traditional database and search tools in reasonable amount of time. Big Data information explosion is mainly due to the vast amount of data generated by social media platform, data input from omni-channels, various mobile devices, user agents, multimedia data, and so on. Overall it is an expanding “Digital Universe”. Big Data predominately revolve around 3V’s: Volume, Velocity, Variety. Big data plays a major role in oil and gas industries. The purpose of this research is to analyze the various problems faced by the Oil & Gas industries in monitoring the vast amount of data received from their various units and to provide a suitable solution for monitoring the data. Oil and Gas (O&G) companies – both the operator companies as well as oil field service providers, now have more upstream data than they are processed before. They are at the verge of managing the vast amount of data generated from exploration , drilling or production works. Management of this is essential for effective, productive, and on demand data insight is critical, for decision making within the organization. The major problem faced by industries is the maintenance of data in concern with Exploration and Production processes. In the oil and gas industry, organizations must apply new technologies and processes that will capture and transform raw data into actionable insight to improve asset value and yield while enhancing safety and protecting the environment. Well and field operations are instrumented to capture a holistic view of equipment performance and well productivity data including reservoir, well, facilities and export data. Leading, analytics-driven oil and gas organizations are connecting people with trusted information to predict business outcomes, and to make real-time decisions that help them outperform their competitors. But for many, these breakaway results remain out of reach. Despite the wealth of data and content available today, decision makers are often starved for true insight. IBM’s big data platform provides a scalable, easy to use, secure information management and analytics platform for complex and large-scale analysis and economical storage of drilling production data. Successfully harnessing big data unleashes the potential to achieve three critical objectives: 1. Enhance exploration and production 2. Improve refining and manufacturing efficiency & 3. Optimize global operations With the help of IBM frameworks the O&G companies can adopt Hadoop enabled Big Data solutions for creating integrated digital Oil Field strategy. Hadoop based solutions allow storing, processing, and analyzing these humongous logs on near real time basis. The crux of solution involves processing raw data in its native format to create aggregated views along with understanding of its relationships and patterns and thereby derive meaningful insight for quick decision-making related to reservoir and optimizing the data exploitation using Map Reduce paradigm. HiveQL is a scalable Data Warehouse solution available on Hadoop, which is similar to SQL syntax. Hive internally generates map-reduce jobs that can be executed on Hadoop clusters. Hive in-turn allows overcoming the learning curve associated with the Map-Reduce code generation. With the help of Hadoop and HiveQL framework the Oil & Gas industries can manage the Big data from various units of oil refineries thereby generating a platform
Glob. J. Mul. App. Sci., 1 (2): 52-57, 2013
53
for efficient utilization of the required data related to the companies for creating the excellence of customers in the global market.